Stop the nightly translation re-translating everything, most days - #685

The Translate Docs cron cost 4 minutes on Aug 3-5 and 118-136 minutes
every day from Aug 6-11 — ~750 wasted runner-minutes and six full-corpus passes
through the LLM gateway in six days.

Aug 3,4,5     4 min   cache survived
Aug 6       125 min   FAIL
Aug 7       136 min   FAIL
Aug 8       122 min   ok    <- saved successfully, and Aug 9 was still slow
Aug 9       118 min   ok
Aug 10      123 min   ok
Aug 11      134 min   ok
Aug 12        5 min   FAIL  <- cache HIT, and the run failed *because* of it

Three causes compound. None of them is the translation cache's own logic, which
is sound — isCached validates per entry against the English source hash, and
the top-level sourceHash field is vestigial (written in three places, read in
none).

1. The cache was evicted between runs — dominant, ~105 min/day

ci.yml cached target/ alongside the cargo registry under a combined
actions/cache@v6, which writes a ref-scoped copy from every branch that
misses the exact key. With build output included each copy is 1.5-2.3 GiB, and
five were live at once — PR refs 677, 679, 680, 681 and main — putting the repo
at 11.56 GiB against GitHub's 10 GiB cap, so the store sits permanently in
LRU eviction.

What that evicted was the 13 KB translation cache, read once every 24 hours
and therefore always the least-recently-used entry in the store. Aug 9's jobs
say so directly: Cache not found for input keys: translation-cache-, translation-cache-, the morning after Aug 8 logged Cache saved with key: translation-cache-1adebf3f….

Fixed with the restore/save split build-daemon.yml:117-144 already uses, whose
comment gives the other reason to want it (a PR branch can otherwise write a
poisoned target/ that a release run restores into a published binary).

2. The cache was saved once, at the end of a serial pipeline

The only save sat in consolidate, downstream of both the matrix gate and
mintlify validate. One page failing in one language discarded all fourteen —
Aug 6 lost ~110 minutes of finished translation to a single ko page. Each
language now saves its own fragment in the job that produced it, right after the
step that proved it good. Each fragment is already authoritative for its own
language, so nothing needs merging first; consolidate's merged save stays as the
cross-language fallback.

A miss is also visible now. The old restore key always evaluated to the bare
literal translation-cache- — the file is gitignored, so it is absent at
checkout and hashFiles returns "" for a path matching nothing. Every restore
that ever worked was a restore-keys prefix match, and a total miss looked
exactly like a hit: nothing failed, nothing warned, the job just spent nine
minutes and a full LLM pass.

3. A cache HIT never checked the file exists — this is Aug 12, and it deadlocks

isCached records that a page was translated once, not that it is on disk
now. Translations land on an auto-translate PR branch; while that sits
unmerged, main lacks the files and the cache still says done, so they are never
regenerated — while --update-nav reads the English tree and emits nav
entries for them. mintlify validate then fails on 28 missing files.

Verified live rather than inferred: docs/cli/update.mdx and
docs/cli/migrate.mdx are on main, docs/zh/cli/ has neither, and #682
carrying them is still open.

That is non-convergent: a cache hit writes nothing → validation fails → the
cache is never saved; a full cache miss spends 120 minutes → goes green. There
was no path to a cheap success while #682 stayed open.

⚠️ This one changes translation OUTPUT, not just caching. The first run
re-sends those 28 pages to the model, so the text will not be byte-identical to
what is on #682 — expect one noisy diff there. ~10 minutes, once.

Guarded at all four call sites. cli.ts is the load-bearing one: the batch
path sorts pages into cached/uncached itself and never calls translateMdxPage
for a cached page, so guarding only the translator would have fixed nothing.

Testing

Beyond the suite, the deadlock was reproduced against the real repo with a
realistically seeded cache (every English page marked translated for zh at its
current hash, as a successful Aug 11 run would have left it):

The new unit tests pin both directions — a missing output re-translates, a
present one is still skipped. The second matters as much as the first: without
it a later refactor could satisfy this PR by turning the guard into a cache
bypass, and every run would be a full re-translation with the suite still green.
I also confirmed the first test fails on the pre-fix code rather than
assuming it would.

• bun run test:run — 3458 pass, 10 skipped
• tsc --noEmit — clean
• eslint on every changed file — clean
• both workflow YAMLs re-parsed after editing

Deliberately not included

• Reordering consolidate's validation so the cache save precedes it. Fix 2
  already makes the cache survive that failure, so this is reorder risk for
  little return.
• A delete channel in the merge script. Correctness only, zero minutes.

Expected effect

Steady state returns to the ~4-minute cache-hit day. The first run after merge
re-translates the 28 pages main is missing (~10 minutes, once), and the repo
cache should fall back under the 10 GiB cap as the four PR-scoped cargo copies
age out.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HEwcerc9jE7ZBkfBiYRbep

Hermes review

Summary

No actionable correctness, security, data-safety, or compatibility defects found in the PR diff. The relevant cache, workflow, batch, and single-page paths were inspected.

Changes

• Splits Rust build-cache restore/save so only main writes large Cargo artifacts.
• Adds per-language translation-cache restore/save and cache-miss visibility.
• Treats a translation cache entry as valid only when its output file exists.

Validation

None configured.

Findings

None.

Open questions

None.

Policy overrides

None.

Summary by CodeRabbit

• Bug Fixes

  • Improved translation caching by detecting missing translated files and regenerating them automatically.
  • Added clearer warnings when translation caches are unavailable.
  • Improved reliability of translated documentation and README generation.

• Chores

  • Updated CI caching to preserve translation and build results more reliably.
  • Extended translation artifact retention to seven days.

• Documentation

  • Added release notes for version 1.0.1-beta.0.

• Tests

  • Added coverage for missing and existing translation outputs to verify cache behavior.